Comparing the performance of statistical methods that generalize effect estimates from randomized controlled trials to much larger target populations.

Policymakers use results from randomized controlled trials to inform decisions about whether to implement treatments in target populations. Various methods - including inverse probability weighting, outcome modeling, and Targeted Maximum Likelihood Estimation - that use baseline data available in both the trial and target population have been proposed to generalize the trial treatment effect estimate to the target population. Often the target population is significantly larger than the trial sample, which can cause estimation challenges. We conduct simulations to compare the performance of these methods in this setting. We vary the size of the target population, the proportion of the target population selected into the trial, and the complexity of the true selection and outcome models. All methods performed poorly when the trial size was only 2% of the target population size or the target population included only 1,000 units. When the target population or the proportion of units selected into the trial was larger, some methods, such as outcome modeling using Bayesian Additive Regression Trees, performed well. We caution against generalizing using these existing approaches when the target population is much larger than the trial sample and advocate future research strives to improve methods for generalizing to large target populations.

Calibrating validation samples when accounting for measurement error in intervention studies.

Many lifestyle intervention trials depend on collecting self-reported outcomes, such as dietary intake, to assess the intervention's effectiveness. Self-reported outcomes are subject to measurement error, which impacts treatment effect estimation. External validation studies measure both self-reported outcomes and accompanying biomarkers, and can be used to account for measurement error. However, in order to account for measurement error using an external validation sample, an assumption must be made that the inferences are transportable from the validation sample to the intervention trial of interest. This assumption does not always hold. In this paper, we propose an approach that adjusts the validation sample to better resemble the trial sample, and we also formally investigate when bias due to poor transportability may arise. Lastly, we examine the performance of the methods using simulation, and illustrate them using PREMIER, a lifestyle intervention trial measuring self-reported sodium intake as an outcome, and OPEN, a validation study measuring both self-reported diet and urinary biomarkers.

Generalizability of Subgroup Effects.

Generalizability methods are increasingly used to make inferences about the effect of interventions in target populations using a study sample. Most existing methods to generalize effects from sample to population rely on the assumption that subgroup-specific effects generalize directly. However, researchers may be concerned that in fact subgroup-specific effects differ between sample and population. In this brief report, we explore the generalizability of subgroup effects. First, we derive the bias in the sample average treatment effect estimator as an estimate of the population average treatment effect when subgroup effects in the sample do not directly generalize. Next, we present a Monte Carlo simulation to explore bias due to unmeasured heterogeneity of subgroup effects across sample and population. Finally, we examine the potential for bias in an illustrative data example. Understanding the generalizability of subgroup effects may lead to increased use of these methods for making externally valid inferences of treatment effects using a study sample.

Perceived Interruptions to HIV Prevention and Treatment Services Associated With COVID-19 for Gay, Bisexual, and Other Men Who Have Sex With Men in 20 Countries.

BACKGROUND: The coronavirus pandemic has necessitated a range of population-based measures to stem the spread of infection. These measures may be associated with disruptions to other health services including for gay, bisexual, and other men who have sex with men (MSM) at risk for or living with HIV. Here, we assess the relationship between stringency of COVID-19 control measures and interruptions to HIV prevention and treatment services for MSM. SETTING: Data for this study were collected between April 16, 2020, and May 24, 2020, as part of a COVID-19 Disparities Survey implemented by the gay social networking app, Hornet. Pandemic control measures were quantified using the Oxford Government Response Tracker Stringency Index: each country received a score (0-100) based on the number and strictness of 9 indicators related to restrictions, closures, and travel bans. METHODS: We used a multilevel mixed-effects generalized linear model with Poisson distribution to assess the association between stringency of pandemic control measures and access to HIV services. RESULTS: A total of 10,654 MSM across 20 countries were included. Thirty-eight percent (3992/10,396) reported perceived interruptions to in-person testing, 55% (5178/9335) interruptions to HIV self-testing, 56% (5171/9173) interruptions to pre-exposure prophylaxis, and 10% (990/9542) interruptions to condom access. For every 10-point increase in stringency, there was a 3% reduction in the prevalence of perceived access to in-person testing (aPR: 0·97, 95% CI: [0·96 to 0·98]), a 6% reduction in access to self-testing (aPR: 0·94, 95% CI: [0·93 to 0·95]), and a 5% reduction in access to pre-exposure prophylaxis (aPR: 0·95, 95% CI: [0·95 to 0·97]). Among those living with HIV, 20% (218/1105) were unable to access their provider; 65% (820/1254) reported being unable to refill their treatment prescription remotely. CONCLUSIONS: More stringent responses were associated with decreased perceived access to services. These results support the need for increasing emphasis on innovative strategies in HIV-related diagnostic, prevention, and treatment services to minimize service interruptions during this and potential future waves of COVID-19 for gay men and other MSM at risk for HIV acquisition and transmission.

Immune-stimulating antibody conjugates elicit robust myeloid activation and durable antitumor immunity.

Innate pattern recognition receptor agonists, including Toll-like receptors (TLRs), alter the tumor microenvironment and prime adaptive antitumor immunity. However, TLR agonists present toxicities associated with widespread immune activation after systemic administration. To design a TLR-based therapeutic suitable for systemic delivery and capable of safely eliciting tumor-targeted responses, we developed immune-stimulating antibody conjugates (ISACs) comprising a TLR7/8 dual agonist conjugated to tumor-targeting antibodies. Systemically administered human epidermal growth factor receptor 2 (HER2)-targeted ISACs were well tolerated and triggered a localized immune response in the tumor microenvironment that resulted in tumor clearance and immunological memory. Mechanistically, ISACs required tumor antigen recognition, Fcγ-receptor-dependent phagocytosis and TLR-mediated activation to drive tumor killing by myeloid cells and subsequent T-cell-mediated antitumor immunity. ISAC-mediated immunological memory was not limited to the HER2 ISAC target antigen since ISAC-treated mice were protected from rechallenge with the HER2- parental tumor. These results provide a strong rationale for the clinical development of ISACs.

Economic, Mental Health, HIV Prevention and HIV Treatment Impacts of COVID-19 and the COVID-19 Response on a Global Sample of Cisgender Gay Men and Other Men Who Have Sex with Men.

There is an urgent need to measure the impacts of COVID-19 among gay men and other men who have sex with men (MSM). We conducted a cross-sectional survey with a global sample of gay men and other MSM (n = 2732) from April 16, 2020 to May 4, 2020, through a social networking app. We characterized the economic, mental health, HIV prevention and HIV treatment impacts of COVID-19 and the COVID-19 response, and examined whether sub-groups of our study population are disproportionately impacted by COVID-19. Many gay men and other MSM not only reported economic and mental health consequences, but also interruptions to HIV prevention and testing, and HIV care and treatment services. These consequences were significantly greater among people living with HIV, racial/ethnic minorities, immigrants, sex workers, and socio-economically disadvantaged groups. These findings highlight the urgent need to mitigate the negative impacts of COVID-19 among gay men and other MSM.

RESUMEN: Existe una necesidad urgente para medir los impactos de COVID-19 entre hombres gay y otros hombres que tienen sexo con hombres (HSH). Hemos conducido una encuesta multifuncional con una prueba mundial de hombres gay y otros HSH (n = 2732) desde el 16 de Abril hasta el 4 de Mayo del 2020, a través de una aplicación de red social. Nosotros caracterizamos los impactos económicos, de salud mental, prevención del VIH y tratamiento del VIH e impactos a COVID-19 y la respuesta de COVID-19, y examinamos si subgrupos de nuestra población de estudio fueron impactados desproporcionadamente por COVID-19. Muchos hombres no tan solo reportaron consecuencias económicas y de salud mental, sino también interrupciones de prevención y de pruebas de VIH, y cuidado del VIH y servicios de tratamiento. Encontramos consecuencias más significantes entre personas viviendo con VIH, grupos raciales/etnicos, migrantes, sexo servidores, y groupos socioeconomicamente disfavorecidos. Los resultados subrayan la necesidad crucial de mitigar los impactos multifacéticos de COVID-19 entre los hombres homosexuales y otros HSH, especialmente para aquellos con vulnerabilidades entrelazadas.

Generalizing randomized trial findings to a target population using complex survey population data.

Randomized trials are considered the gold standard for estimating causal effects. Trial findings are often used to inform policy and programming efforts, yet their results may not generalize well to a relevant target population due to potential differences in effect moderators between the trial and population. Statistical methods have been developed to improve generalizability by combining trials and population data, and weighting the trial to resemble the population on baseline covariates. Large-scale surveys in fields such as health and education with complex survey designs are a logical source for population data; however, there is currently no best practice for incorporating survey weights when generalizing trial findings to a complex survey. We propose and investigate ways to incorporate survey weights in this context. We examine the performance of our proposed estimator through simulations in comparison to estimators that ignore the complex survey design. We then apply the methods to generalize findings from two trials-a lifestyle intervention for blood pressure reduction and a web-based intervention to treat substance use disorders-to their respective target populations using population data from complex surveys. The work highlights the importance in properly accounting for the complex survey design when generalizing trial findings to a population represented by a complex survey sample.

Target validity: Bringing treatment of external validity in line with internal validity.

PURPOSE OF REVIEW: "Target bias" is the difference between an estimate of association from a study sample and the causal effect in the target population of interest. It is the sum of internal and external bias. Given the extensive literature on internal validity, here, we review threats and methods to improve external validity. RECENT FINDINGS: External bias may arise when the distribution of modifiers of the effect of treatment differs between the study sample and the target population. Methods including those based on modeling the outcome, modeling sample membership, and doubly robust methods are available, assuming data on the target population is available. SUMMARY: The relevance of information for making policy decisions is dependent on both the actions that were studied and the sample in which they were evaluated. Combining methods for addressing internal and external validity can improve the policy relevance of study results.

Implementing statistical methods for generalizing randomized trial findings to a target population.

Randomized trials are considered the gold standard for assessing the causal effects of a drug or intervention in a study population, and their results are often utilized in the formulation of health policy. However, there is growing concern that results from trials do not necessarily generalize well to their respective target populations, in which policies are enacted, due to substantial demographic differences between study and target populations. In trials related to substance use disorders (SUDs), especially, strict exclusion criteria make it challenging to obtain study samples that are fully "representative" of the populations that policymakers may wish to generalize their results to. In this paper, we provide an overview of post-trial statistical methods for assessing and improving upon the generalizability of a randomized trial to a well-defined target population. We then illustrate the different methods using a randomized trial related to methamphetamine dependence and a target population of substance abuse treatment seekers, and provide software to implement the methods in R using the "generalize" package. We discuss several practical considerations for researchers who wish to utilize these tools, such as the importance of acquiring population-level data to represent the target population of interest, and the challenges of data harmonization.

Sensitivity analyses for effect modifiers not observed in the target population when generalizing treatment effects from a randomized controlled trial: Assumptions, models, effect scales, data scenarios, and implementation details.

BACKGROUND: Randomized controlled trials are often used to inform policy and practice for broad populations. The average treatment effect (ATE) for a target population, however, may be different from the ATE observed in a trial if there are effect modifiers whose distribution in the target population is different that from that in the trial. Methods exist to use trial data to estimate the target population ATE, provided the distributions of treatment effect modifiers are observed in both the trial and target population-an assumption that may not hold in practice. METHODS: The proposed sensitivity analyses address the situation where a treatment effect modifier is observed in the trial but not the target population. These methods are based on an outcome model or the combination of such a model and weighting adjustment for observed differences between the trial sample and target population. They accommodate several types of outcome models: linear models (including single time outcome and pre- and post-treatment outcomes) for additive effects, and models with log or logit link for multiplicative effects. We clarify the methods' assumptions and provide detailed implementation instructions. ILLUSTRATION: We illustrate the methods using an example generalizing the effects of an HIV treatment regimen from a randomized trial to a relevant target population. CONCLUSION: These methods allow researchers and decision-makers to have more appropriate confidence when drawing conclusions about target population effects.

Generalizability of randomized trial results to target populations: Design and analysis possibilities.

Randomized trials play an important role in estimating the effect of a policy or social work program in a given population. While most trial designs benefit from strong internal validity, they often lack external validity, or generalizability, to the target population of interest. In other words, one can obtain an unbiased estimate of the study sample average treatment effect (SATE) from a randomized trial; however, this estimate may not equal the target population average treatment effect (TATE) if the study sample is not fully representative of the target population. This paper provides an overview of existing strategies to assess and improve upon the generalizability of randomized trials, both through statistical methods and study design, as well as recommendations on how to implement these ideas in social work research.

Measuring Model Misspecification: Application to Propensity Score Methods with Complex Survey Data.

Model misspecification is a potential problem for any parametric-model based analysis. However, the measurement and consequences of model misspecification have not been well formalized in the context of causal inference. A measure of model misspecification is proposed, and the consequences of model misspecification in non-experimental causal inference methods are investigated. The metric is then used to explore which estimators are more sensitive to misspecification of the outcome and/or treatment assignment model. Three frequently used estimators of the treatment effect are considered, all of which rely on the propensity score: (1) full matching, (2) 1:1 nearest neighbor matching, and (3) weighting. The performance of these estimators is evaluated under two different sampling designs: (1) simple random sampling (SRS) and (2) a two-stage stratified survey. As the degree of misspecification of either the propensity score or outcome model increases, so does the bias and the root mean square error, while the coverage decreases. Results are similar for the simple random sample and a complex survey design.

HIV prevalence and behavioral and psychosocial factors among transgender women and cisgender men who have sex with men in 8 African countries: A cross-sectional analysis.

INTRODUCTION: Sub-Saharan Africa bears more than two-thirds of the worldwide burden of HIV; however, data among transgender women from the region are sparse. Transgender women across the world face significant vulnerability to HIV. This analysis aimed to assess HIV prevalence as well as psychosocial and behavioral drivers of HIV infection among transgender women compared with cisgender (non-transgender) men who have sex with men (cis-MSM) in 8 sub-Saharan African countries. METHODS AND FINDINGS: Respondent-driven sampling targeted cis-MSM for enrollment. Data collection took place at 14 sites across 8 countries: Burkina Faso (January-August 2013), Côte d'Ivoire (March 2015-February 2016), The Gambia (July-December 2011), Lesotho (February-September 2014), Malawi (July 2011-March 2012), Senegal (February-November 2015), Swaziland (August-December 2011), and Togo (January-June 2013). Surveys gathered information on sexual orientation, gender identity, stigma, mental health, sexual behavior, and HIV testing. Rapid tests for HIV were conducted. Data were merged, and mixed effects logistic regression models were used to estimate relationships between gender identity and HIV infection. Among 4,586 participants assigned male sex at birth, 937 (20%) identified as transgender or female, and 3,649 were cis-MSM. The mean age of study participants was approximately 24 years, with no difference between transgender participants and cis-MSM. Compared to cis-MSM participants, transgender women were more likely to experience family exclusion (odds ratio [OR] 1.75, 95% CI 1.42-2.16, p < 0.001), rape (OR 1.95, 95% CI 1.63-2.36, p < 0.001), and depressive symptoms (OR 1.30, 95% CI 1.12-1.52, p < 0.001). Transgender women were more likely to report condomless receptive anal sex in the prior 12 months (OR 2.44, 95% CI 2.05-2.90, p < 0.001) and to be currently living with HIV (OR 1.81, 95% CI 1.49-2.19, p < 0.001). Overall HIV prevalence was 25% (235/926) in transgender women and 14% (505/3,594) in cis-MSM. When adjusted for age, condomless receptive anal sex, depression, interpersonal stigma, law enforcement stigma, and violence, and the interaction of gender with condomless receptive anal sex, the odds of HIV infection for transgender women were 2.2 times greater than the odds for cis-MSM (95% CI 1.65-2.87, p < 0.001). Limitations of the study included sampling strategies tailored for cis-MSM and merging of datasets with non-identical survey instruments. CONCLUSIONS: In this study in sub-Saharan Africa, we found that HIV burden and stigma differed between transgender women and cis-MSM, indicating a need to address gender diversity within HIV research and programs.

Evidence for contribution of common genetic variants within chromosome 8p21.2-8p21.1 to restricted and repetitive behaviors in autism spectrum disorders.

BACKGROUND: Restricted and Repetitive Behaviors (RRB), one of the core symptom categories for Autism Spectrum Disorders (ASD), comprises heterogeneous groups of behaviors. Previous research indicates that there are two or more factors (subcategories) within the RRB domain. In an effort to identify common variants associated with RRB, we have carried out a genome-wide association study (GWAS) using the Autism Genetic Resource Exchange (AGRE) dataset (n = 1,335, all ASD probands of European ancestry) for each identified RRB subcategory, while allowing for comparisons of associated single nucleotide polymorphisms (SNPs) with associated SNPs in the same set of probands analyzed using all the RRB subcategories as phenotypes in a multivariate linear mixed model. The top ranked SNPs were then explored in an independent dataset. RESULTS: Using principal component analysis of item scores obtained from Autism Diagnostic Interview-Revised (ADI-R), two distinct subcategories within Restricted and Repetitive Behaviors were identified: Repetitive Sensory Motor (RSM) and Insistence on Sameness (IS). Quantitative RSM and IS scores were subsequently used as phenotypes in a GWAS using the AGRE ASD cohort. Although no associated SNPs with genome-wide significance (P < 5.0E-08) were detected when RSM or IS were analyzed independently, three SNPs approached genome-wide significance when RSM and IS were considered together using multivariate association analysis. These included the top IS-associated SNP, rs62503729 (P-value = 6.48E-08), which is located within chromosome 8p21.2-8p21.1, a locus previously linked to schizophrenia. Notably, all of the most significantly associated SNPs are located in close proximity to STMN4 and PTK2B, genes previously shown to function in neuron development. In addition, several of the top-ranked SNPs showed correlations with STMN4 mRNA expression in adult CEU (Caucasian and European descent) human prefrontal cortex. However, the association signals within chromosome 8p21.2-8p21.1 failed to replicate in an independent sample of 2,588 ASD probands; the insufficient sample size and between-study heterogeneity are possible explanations for the non-replication. CONCLUSIONS: Our analysis indicates that RRB in ASD can be represented by two distinct subcategories: RSM and IS. Subsequent univariate and multivariate genome-wide association studies of these RRB subcategories enabled the detection of associated SNPs at 8p21.2-8p21.1. Although these results did not replicate in an independent ASD dataset, genomic features of this region and pathway analysis suggest that common variants in 8p21.2-8p21.1 may contribute to RRB, particularly IS. Together, these observations warrant future studies to elucidate the possible contributions of common variants in 8p21.2-8p21.1 to the etiology of RSM and IS in ASD.

The Complex Epidemiology of Carbapenem-Resistant Enterobacter Infections: A Multicenter Descriptive Analysis.

BACKGROUND: The pandemic of carbapenem-resistant Enterobacteriaceae (CRE) was primarily due to clonal spread of bla KPC producing Klebsiella pneumoniae. Thus, thoroughly studied CRE cohorts have consisted mostly of K. pneumoniae. OBJECTIVE: To conduct an extensive epidemiologic analysis of carbapenem-resistant Enterobacter spp. (CREn) from 2 endemic and geographically distinct centers. METHODS: CREn were investigated at an Israeli center (Assaf Harofeh Medical Center, January 2007 to July 2012) and at a US center (Detroit Medical Center, September 2008 to September 2009). bla KPC genes were queried by polymerase chain reaction. Repetitive extragenic palindromic polymerase chain reaction and pulsed-field gel electrophoresis were used to determine genetic relatedness. RESULTS: In this analysis, 68 unique patients with CREn were enrolled. Sixteen isolates (24%) were from wounds, and 33 (48%) represented colonization only. All isolates exhibited a positive Modified Hodge Test, but only 93% (27 of 29) contained bla KPC. Forty-three isolates (63%) were from elderly adults, and 5 (7.4%) were from neonates. Twenty-seven patients died in hospital (40.3% of infected patients). Enterobacter strains consisted of 4 separate clones from Assaf Harofeh Medical Center and of 4 distinct clones from Detroit Medical Center. CONCLUSIONS: In this study conducted at 2 distinct CRE endemic regions, there were unique epidemiologic features to CREn: (i) polyclonality, (ii) neonates accounting for more than 7% of cohort, and (iii) high rate of colonization (almost one-half of all cases represented colonization). Since false-positive Modified Hodge Tests in Enterobacter spp. are common, close monitoring of carbapenem resistance mechanisms (particularly carbapenemase production) among Enterobacter spp. is important.